Typically, random numbers (e.g., pseudo random number) are utilized as inputs for performing dropout and weight initialization in a neural network. Correlation and the unexpected period of the random numbers could make learning (or training) of the neural network inefficient. This can be addressed by performing batch normalization on the input random numbers.
However, the batch normalization requires extra computation efforts for generating Gaussian probability distribution of random number signals, and thus, causes much computational time penalty of, e.g., about 30%.
Thus, there is a need for a true-random number generator providing random numbers with less correlation or expected period thereof to be used for performing dropout or weight initialization in the neural network.